System and method for adaptive positioning of a subject for capturing a thermal image

ABSTRACT

A method for determining a view angle of a thermal image from a user and generating a suggestion to enable the user for adaptive positioning of a subject for capturing the thermal image is provided. The method includes (i) receiving a thermal image of a body of a subject, (ii) automatically determining a view angle of the thermal image from a user using a view angle estimator, (iii) determining an angular adjustment to be made to a view position of the thermal imaging camera or a position of the subject by comparing the determined view angle with a required view angle as per thermal imaging protocol when the thermal image does not meet the required view angle and (iv) generating instructions to the user for adjusting the view position of the thermal imaging camera for capturing a new thermal image at the required view angle as per thermal imaging protocol.

BACKGROUND Technical Field

The present invention is directed towards capturing thermal imageconformant to standard operating procedure and, more particularly, to asystem and method for determining a view angle of a thermal image of asubject from a user to generate a suggestion to capture the thermalimage as per standard breast thermography protocol by enabling the usefor adaptive positioning of the subject and capturing.

Description of the Related Art

Breast Cancer is among the leading cause of cancer deaths around theworld especially in women. Mammography is considered as gold standardfor detecting breast cancer. However, it is very costly, painful due tothe compression of breast and has a radiation exposure. In the recentyears, thermography is emerging as a promising modality for breastcancer screening. Thermography captures the amount of heat radiatingfrom the surface of the body and measures the temperature patterns anddistribution on the chest due to high metabolism associated withtumorous growth. There are several advantages of Breast thermographycompared to other methods. The breast thermography works on women of allage groups, does not involve any radiation and non-contact and hencepainless. The key challenge in breast thermography is thepreconditioning and capturing of thermal images of the patient. As tumorcannot be captured always using a single fixed view, the medical expertrequires multiple thermal views/videos of the patient for analysis. Thismultiple view capture is usually done manually by either moving thecamera or making the patients turn to required view angles. This kind ofcapturing requires high expertise and it is observed that low to mediumskilled technicians/thermographers are finding difficulty in capturingthese thermal images at specified angles due to lack of proper guidance.This also leads to non-standardisation of data across subjects/patientsand even may lead to poor performance of breast analysis involvingbreast segmentation and classification. Typical protocol for breastthermography is to capture five views as five image files named manuallyby technician. This manual file labelling is error prone and can resultin incorrect classification due to wrong image mapping.

Hence, there is a need for an automated guidance system/method toautomatically predict the view angle and guide the technician forcapturing proper thermal images at required view angles. This can alsohelp in enabling labelling of each frame in the video according to theirview angle.

SUMMARY

In view of the foregoing, embodiment herein provides a method fordetermining a angle of a thermal image from a user and generating asuggestion to enable the user for adaptive positioning of a subject forcapturing the thermal image. The method includes (i) receiving a thermalimage of a body of a subject, which represents the temperaturedistribution on the body of the subject as pixels in the thermal imagewith a highest temperature value being displayed in a first color andpixels with a lowest temperature value being displayed in a secondcolor, pixels with temperature values between the lowest and highesttemperature values being displayed in gradations of color between thefirst and second colors; (ii) automatically determining a view angle ofthe thermal image from a user using a view angle estimator; (iii)determining, using a machine learning model, an angular adjustment to bemade to a view position of the thermal imaging camera or a position ofthe subject by comparing the determined view angle with a required viewangle as per thermal imaging protocol when the thermal image does notmeet the required vie angle as per thermal imaging protocol; and (iv)generating a set of instructions to the user for adjusting the viewposition of the thermal imaging camera for capturing a new thermal imageat the required view angle as per thermal imaging protocol. The thermalimage is captured by a thermal imaging camera that includes an array ofsensors, a lens and a specialized processor. The array of sensorsconverts infrared energy into electrical signals on a per-pixel basis.The lens focuses the infrared energy from the subject's body onto thearray of sensors. The array of sensors detects temperature values fromthe subject's body. The specialized processor processes the detectedtemperature values into at least one block of pixels to generate thethermal image.

In an embodiment, the method includes the step of providing the newcaptured thermal image along with determined view angle for automaticbreast segmentation followed by an automatic tumor detection method todetect cancerous tissue and/or non-cancerous tissue within a breast areaof the subject.

Its another embodiment, the method includes, step of implementing anautomatic segmenting technique to segment the breast area of the subjectin the thermal image by (i) determining an outer side contour of anoutline of a boundary of the breast area of the subject from a bodysilhouette, (ii) determining an inner side boundary of the breast areafrom the body silhouette and the view angle of the thermal image, (iii)determining an upper boundary of the breast area by determining a lowerboundary of an isotherm of axilla of the subject, (iv) determining alower boundary of the breast area by determining an upper boundary of anisotherm of infra-mammary fold of the person and (v) segmenting thebreast area of the, subject by connecting above determined breastboundaries to segment the breast from surrounding tissue in the thermalimage.

In yet another embodiment, the implementation of the automaticsegmenting technique to segment the breast area of the subject in thethermal image includes (i) training a deep learning model by providing aplurality of thermal images as an input and the correspondingsegmentation as an output to obtain the trained deep learning model and(ii) providing the new captured thermal image to the trained deeplearning model to predict a final segmentation map.

In yet another embodiment, the view angle estimator includes aregression machine learning model that determines an approximate viewangle of the subject with respect to the camera using the thermal image.

In yet another embodiment, the view angle estimator includes a taggingclassifier. The tagging classifier includes a machine learning modelthat determines the view of the thermal image and classifies the thermalimage as one of the discrete views such as a right lateral view, a rightoblique view, a frontal view, a left lateral view or a left oblique viewas per the thermal imaging protocol.

In yet another embodiment, the set of instructions is provided to atleast one of a robotic arm holding the camera, an electronicallycontrolled camera stand or an electronically controlled rotating chairto automatically positions itself to the suggested angular adjustmentfor capturing the new thermal image at the required view angle.

In yet another embodiment, the view angle estimator automaticallycaptures the new thermal image at the required view angle without user'sintervention.

In yet another embodiment, the method includes the step of displaying atleast one of the determined view angle of the thermal image or thesegmented breast area on a visualization screen.

In yet another embodiment, the automatic tumor detection method includes(i) determining a percentage of pixels p₁ within said selected region ofinterest with a temperature T¹ _(pixel), where T₂≤T¹ _(pixel)≤T₁m, (ii)determining a percentage of pixels p₂ within said selected region ofinterest with a temperature T² _(pixel), where T₃≤T² _(pixel) and (iii)determining a ratio p₃=P_(edge)/P_(block), where P_(edge) is a number ofpixels around a border of a suspected tumor within said region ofinterest, and P_(block) is a number of pixels in the perimeter of saidregion of interest. The T₁, T₂ and T₃ are temperature threshold obtainedfrom temperature distribution

In yet another embodiment, the method includes the step of determiningwhether a suspected tumor region as one of: said cancerous tissue, saidnon-cancerous using a decision rule R. The decision rule R is based onany combination of: R₁, R₂, R₃, where: R₁=(p₁>Threshold₁),R₂=(p₂>Threshold₂), and R₃=(p₃>Threshold₃).

In yet another embodiment, the view angle estimator automatically labelsthe thermal image with a label signifying its discrete view angle. Inyet another embodiment, the method comprises the step of correcting viewangle errors in said thermal imaging protocol by modifying the dataobject properties comprising filenames using said label.

In yet another embodiment, the view angle estimator is applied on athermal image that is obtained by selecting a single image frame of athermal video or a live stream thermal video. The thermal video or thelive stream thermal video is captured using the thermal imaging camera.

In yet another embodiment, the view angle estimator includes a taggingclassifier that selects a thermal image that meets the required viewangle from one or more thermal images that are obtained from a thermalvideo or live stream thermal video.

In yet another embodiment, the set of instructions includes at least oneof a text, a visual or audio for capturing the new thermal image at therequired view angle.

In another aspect, a system for determining a view angle of a thermalimage from a user and generating a suggestion to enable the user foradaptive positioning of a subject for capturing the thermal image isprovided. The system includes a storage device, and a processorretrieving machine-readable instructions from the storage device which,when executed by the processor, enable the processor to (i) receive athermal image of a body of a subject, which represents the temperaturedistribution on the body of the subject as pixels in the thermal imagewith a highest temperature value being displayed in a first color andpixels with a lowest temperature value being displayed in a secondcolor, pixels with temperature values between the lowest and highesttemperature values being displayed in gradations of color between thefirst and second colors; (ii) automatically determine a view angle ofthe thermal image; (iii) determine an angular adjustment to be made to aview position of the thermal imaging camera or a position of the subjectby comparing the determined view angle with a required view angle as perthermal imaging protocol when the thermal image does not meet therequired view angle as per thermal imaging protocol, using a machinelearning model; (iv) generate a set of instructions to the user foradjusting the view position of the thermal imaging camera for capturinga new thermal image at the required view angle as per thermal imagingprotocol. The thermal imaging camera includes an array of sensors, alens and a specialized processor. The array of sensors converts infraredenergy into electrical signals on a per-pixel basis. The lens focusesthe infrared energy from the subject's body onto the array of sensors.The array of sensors detects temperature values from the subject's body.The specialized processor processes the detected temperature values intoat least one block of pixels to generate the thermal image.

In an embodiment, the system provides the new captured thermal imagealong with determined view angle to the view angle determination unitfor automatic breast segmentation followed by an automatic tumordetection method to detect cancerous tissue and/or non-cancerous tissuewithin a breast area of the subject.

In another embodiment, the system implements an automatic segmentingtechnique to segment the breast area of the subject in the thermal imageby (i) determining an outer side contour of an outline of a boundary ofthe breast area of the subject from a body silhouette, (ii) determiningan inner side boundary of the breast area from the body silhouette andthe view angle of the thermal image, (iii) determining an upper boundaryof the breast area by determining a lower boundary of an isotherm ofaxilla of the subject, (iv) determining a lower boundary of the breastarea by determining an upper boundary of an isotherm of infra-mammaryfold of the person and (v) segmenting the breast area of the subject byconnecting above determined breast boundaries to segment the breast fromsurrounding tissue in the thermal image.

In another embodiment, the system implements the automatic segmentingtechnique to segment the breast area of the subject in the thermal imageincludes training a deep learning model by providing a plurality ofthermal images as an input and the corresponding segmentation as anoutput to obtain the trained deep learning model and providing the newcaptured thermal image to the trained deep learning model to predict afinal segmentation map.

In yet another embodiment, the system provides set of instructions to atleast one of a robotic arm holding the camera, an electronicallycontrolled camera stand or an electronically controlled rotating chairto automatically positions itself to the suggested angular adjustmentfor capturing the new thermal image at the required view angle.

The system ensures correct segmentation of the breasts region withbetter accuracy. The system enables automatic selection of requiredviews from the videos and guides a technician to capture the perfectview of the thermal image. The system may automate the thermal imagecapturing by obtaining feedback from the tagging classifier/the viewangle estimator. A set of frames from a video may be passed as a batchinput to the system and the system may predict a view angle to enablesegmentation of the breast region in all frames. The system performs anautomated image capturing with minimal or no human intervention duringimage capture.

These and other aspects of the embodiments: herein will be betterappreciated and understood when considered in conjunction with thefollowing description and the accompanying drawings. It should beunderstood, however, that the following descriptions, while indicatingpreferred embodiments and numerous specific details thereof, are givenby way of illustration and not of limitation. Many changes andmodifications may be made within the scope of the embodiments hereinwithout departing from the spirit thereof, and the embodiments hereininclude all such modifications.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments herein will, be better understood from the followingdetailed description with reference to the drawings, in which:

FIG. 1 illustrates an example female patient with a thermal imagingcamera mounted on a slidable and axially rotatable robotic arm formoving the thermal camera along a semi-circular trajectory fromside-to-side in front of the patient according to an embodiment herein;

FIG. 2 illustrates an exploded view of an adaptive view anglepositioning system for determining a view angle of a thermal image froma user and for generating a suggestion to enable the user for adaptivepositioning of a subject for capturing the thermal image according to anembodiment herein;

FIG. 3 illustrates an exemplary process flow of an automated Region ofInterest (ROI) analysis of a thermal image from a user using an adaptiveview angle positioning system according to an embodiment herein;

FIG. 4 illustrates an exemplary process flow of an offline view angledetermination of a thermal image from a user using an adaptive viewangle positioning system according to an embodiment herein;

FIG. 5 illustrates an exemplary process flow of a live stream view angledetermination of a thermal image from a user using an adaptive viewangle positioning system according to an embodiment herein;

FIG. 6 illustrates an exemplary process flow of a view angledetermination using an adaptive view angle positioning system to selecta view with adaptive sampling to reduce image frame candidates accordingto an embodiment herein;

FIG. 7 illustrates a flow diagram of one embodiment of the presentmethod for determining a view angle of a thermal image from a user andgenerating a suggestion to enable the user for adaptive positioning of asubject for capturing the thermal image according to an embodimentherein; and

FIG. 8 illustrates a block diagram of one example adaptive view anglepositioning system/image processing system for processing a thermalimage in accordance with the embodiments described with respect to theflow diagram of FIG. 7 according to an embodiment herein.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The embodiments herein and the various, features and advantageousdetails thereof are explained more fully with reference to thenon-lining embodiments that are illustrated in the accompanying drawingsand detailed in the following description, Descriptions of well-knowncomponents and processing techniques are omitted so as to notunnecessarily obscure the embodiments herein. The examples used hereinare intended merely to facilitate an understanding of ways in which theembodiments herein may be practiced and to further enable those of skillin the art to practice the embodiments herein. Accordingly, the examplesshould not be construed as limiting the scope of the embodiments herein.

As mentioned there remains a need for a system, and a method fordetermining a view angle of a thermal image from a user and generating asuggestion to enable the user for adaptive positioning of a subject forcapturing the thermal image. Referring now to the drawings, and moreparticularly to FIGS. 1 through 8 , where similar reference charactersdenote corresponding features consistently throughout the figures, thereare shown preferred embodiments.

A “person” refers to either a male or a female. Gender pronouns are notto be viewed as limiting the scope of the appended claims strictly tofemales. Moreover, although the term “person” or “patient” is usedinterchangeably throughout this disclosure, it should be appreciatedthat the person undergoing breast cancer screening may be somethingother than a human such as, for example, a primate. Therefore, the useof such terms is not to be viewed as limiting the scope of the appendedclaims to humans.

A “breast area” refers to tissue of the breast and may further includesurrounding tissue as is deemed appropriate for breast cancer screening.Thermal images are the capture of the breast area in various view angleswhich include a mediolateral view (center chest), a mediolateral oblique(angular) view, and a lateral (side) view, as are generally understoodin the medical imaging arts. It should be appreciated that themediolateral view is a supplementary mammographic view which generallyshows less breast tissue and pectoral muscle than the mediolateraloblique view. FIG. 1 shows the breast area of a female 100. It should beappreciated that the patient may be stationary while the camera movesabout the patient, or the patient can move while the camera remainsstationary, or the patient and the camera, may move to capture theappropriate view angles as desired.

A “thermal camera” refers to either a still camera or a video camerawith a lens that focuses infrared energy from objects in a scene onto anarray of specialized sensors which convert infrared energy across adesired thermal wavelength band into electrical signals on a per-pixelbasis and which output an array of pixels with colours that correspondto temperatures of the objects in the image.

A “thermographic image” or simply a “thermal image” is an image capturedby a thermal camera. The thermographic image comprises an array of colorpixels with each color being associated with temperature. Pixels with ahigher temperature value are displayed in the thermal image in a firstcolor and pixels with a lower temperature value are displayed in asecond color. Pixels with temperature values between the lower andhigher temperature values are, displayed in gradations of color betweenthe first and second colors.

“Receiving a thermal image” of a patient for cancer screening isintended to be widely construed and includes retrieving, capturing,acquiring, or otherwise obtaining video image frames.

“Analyzing the thermographic image” means to identify a plurality ofpoints (PN) in the image.

A “software interface tool” is a composite of functionality for tumordetection and/or tumor classification using a plurality ofuser-selectable objects displayed on a display device such as atouchscreen display. One embodiment of a software interface tool whichimplements a tumor detection method is disclosed in commonly owned andco-pending U.S. patent application Ser. No. 14/668,178, entitled:“Software Interface Tool For Breast Cancer Screening” by KrithikaVenkataramani et al. Another embodiment of a software interface toolwhich implements a tumor classification method is disclosed in commonlyowned and co-pending U.S. patent application Ser. No. 15/053,767,entitled: “Software Interface Tool For Breast Cancer Screening”, byGayatri Sivakumar et al. Various embodiments of the software interfacetool perform manual, semi-automatic, and automatic selection of a blockof pixels in the thermal image for screening.

FIG. 1 illustrates an example female patient with a thermal imagingcamera mounted on a slidable and axially rotatable robotic arm formoving the thermal camera along a semi-circular trajectory fromside-to-side in front of the patient according to an embodiment herein.The thermal imaging camera 101 is mounted on a slidable and axiallyrotatable robotic arm 102 capable of moving the thermal imaging cameraalong a semi-circular trajectory 103 in the front of the patient/subjectfrom side-to-side such that thermographic images may be captured in aright-side view 104, a front view 105, and a left-side view 106, andvarious oblique angles in between. The thermal imaging camera 101 can bea single-band infrared camera, a multi-band infrared camera in thethermal range, and a hyperspectral infrared camera in the thermal range.The resolution of the thermal imaging camera 101 is effectively the sizeof the pixel. Smaller pixels mean that the resulting image have a higherresolution and thus better spatial definition. Although the thermalimaging camera 101 offers a relatively large dynamic range oftemperature settings, it is preferable that the camera's temperaturerange be relatively small, centered around the person's body surfacetemperature so that small temperature variations are amplified in termsof pixel color changes in order to provide a better measure oftemperature variation. Thermal imaging cameras are readily available invarious streams of commerce. The thermal imaging camera 101 iscommunicatively connected to an adaptive view angle positioning system107 which process the thermal image captured by the thermal imagingcamera 101 for determining a view angle of the thermal image from a userand for generating a suggestion to enable the user/robotic arm foradaptive positioning of a patient/subject for capturing the thermalimage.

FIG. 2 illustrates an exploded view of an adaptive view anglepositioning system for determining a view angle of a thermal image froma user and for generating a suggestion to enable the user for adaptivepositioning of a subject for capturing the thermal image according to anembodiment herein. The adaptive view angle positioning system 107includes a thermal image receiving module 202, a view angle estimator204, a view angle determination module 206 and a thermal camera controlmodule 208. The thermal image receiving module 202 receives a thermalimage of a body of a subject/patient. The thermal image represents thetemperature distribution on the body of the subject as pixels in thethermal image with a highest temperature value being displayed in afirst color and pixels with a lowest temperature value being displayedin a second color, pixels with temperature values between the lowest andhighest temperature values being displayed in gradations of colorbetween the first and second colors. In an embodiment, the thermal imageis captured using a thermal imaging camera which is connected with theadaptive view angle positioning system 107. In an embodiment, thethermal imaging camera includes an array of sensors, a lens and aspecialized processor. The array of sensors converts infrared energyinto electrical signals on a per-pixel basis. The lens focuses theinfrared energy from the subject's body onto the array of sensors. Thearray of sensors detects temperature values from the subject's body. Thespecialized processor processes the detected temperature values into atleast one block of pixels to generate the thermal image. The view angleestimator 204 automatically determines a view angle of the thermalimage. In an embodiment, the view angle of the thermal image isdetermined using a tagging classifier that selects a thermal image thatmeets the required view angle from one or more thermal images that areobtained from a thermal video or a live stream thermal video. Thetagging classifier includes a machine learning model that determines theview of the thermal image and classifies the thermal image as one of thediscrete views such as a right lateral view, a right oblique view, afrontal view, a left lateral view or a left oblique view as per thethermal imaging protocol. In an embodiment, the thermal imaging protocolincludes at least one steps of (i) cooling the thermal image for aparticular time interval, (ii) positioning the subject in such a waythat the thermal image of the complete chest area with axilla isvisible, (iii) focusing the thermal image of the subject for capturingthe high resolution thermal image of the subject, (iv) capturing thethermal image of the subject in at least one of front view, left obliqueview, left lateral view, right oblique view or right lateral view or (v)providing the captured thermal image of the subject to the system forfurther analysis. The view angle estimator 204 automatically labels thethermal image with a label signifying its discrete view angle. The labelmay include a file name for the thermal image. In an embodiment, theview angle estimator 204 is applied on a thermal image that is obtainedby selecting a single image frame of a thermal video or a live streamthermal video. In another embodiment, the view angle estimator 204includes a regression machine learning model that determines anapproximate view angle of the subject with respect to the thermalimaging camera using the thermal image. The view angle estimator 204includes a convolution neural network regressor to estimate the viewangle directly from the thermal image.

The view angle determination module 206 determines an angular adjustmentto be made to a view position of the thermal imaging camera or aposition of the subject using a machine learning model. The view angledetermination module 206 compares the determined view angle with arequired view angle as per thermal imaging protocol when the thermalimage does not meet the required view angle as per thermal imagingprotocol to determine an angular adjustment to be made to a viewposition of the thermal imaging camera or a position of the subject. Thethermal camera control module 208 provides set of instructions totechnician or at least one of a robotic arm holding the camera, anelectronically controlled camera stand or an electronically controlledrotating chair to automatically positions itself to the suggestedangular adjustment for capturing the new thermal image at the requiredview angle as per thermal image protocol. The set of instructionsincludes at least one of a text, a visual or audio for capturing the newthermal image at the required view angle.

The adaptive view angle positioning system 107 provides the new capturedthermal image along with determined view angle for automatic breastsegmentation followed by an automatic tumor detection method to detectcancerous tissue and/or non-cancerous tissue within a breast area of thesubject. In an, embodiment, the view angle estimator automaticallycaptures the new thermal image at the required view angle without user'sintervention. In an embodiment, the adaptive view angle positioningsystem 107 adapted with a segmentation system or module that implementsan automatic segmenting technique to segment the breast area of thesubject in the thermal image by determining at least one of (i) an outerside contour of an outline of a boundary of the breast area of thesubject from a body silhouette (ii) an inner side boundary of the breastarea from the body silhouette and the view angle of the thermal image,(iii) an upper boundary of the breast area by determining a lowerboundary of an isotherm of axilla of the subject, or (iv) a lowerboundary of the breast area by determining an upper boundary of anisotherm of infra-mammary fold of the person. The adaptive view anglepositioning system 107 segments the breast area of the subject byconnecting above determined breast boundaries to segment the breast fromsurrounding tissue in the thermal image.

In an embodiment, the adaptive view angle positioning system 107 mayincludes a machine learning model for automatically segmenting thebreast area of the subject in the thermal image. The machine learningmodel is trained by providing a plurality of thermal images as an inputand the corresponding segmentation as an output to obtain the traineddeep learning model. The adaptive view angle positioning system 107provides the new captured thermal image to the trained deep learningmodel to predict a final segmentation map. The adaptive view anglepositioning system 107 may display at least one of the determined viewangle of the thermal image or the segmented breast area on avisualization screen. In another embodiment, the automatic tumordetection includes the steps of (i) determining a percentage of pixelspi within said selected region of interest with a temperature T¹_(pixel), where T₂≤T¹ _(pixel)≤T₁, (ii) determining a percentage ofpixels p₂ within said selected region of interest with a temperature T²_(pixel), where T₃≤T² _(pixel) and (iii) determining a ratiop₃=P_(edge)/P_(block), where P_(edge) is a number of pixels around aborder of a suspected tumor within said region of interest, andP_(block) is a number of pixels in the perimeter of the region ofinterest. The adaptive view angle positioning system 107 includes amalignancy classifier that determines whether a suspected tumor regionas one of: said cancerous tissue, said non-cancerous using a decisionrule R. The decision R is based on any combination of: R₁, R₂, R₃,where: R₁=(p₁>Threshold₁), R₂=(p₂>Threshold₂), and R₃=(p₃>Threshold₃).

FIG. 3 illustrates an exemplary process flow of automated Region ofInterest (ROI) analysis of a thermal image from a user using an adaptiveview angle positioning system according to an embodiment herein. At step302, the thermal image is captured using a thermal imaging camera. Insome embodiment, the thermal image may be received or retrieved from aremote device over a network, or from a media such as a CDROM or DVD.The thermal image may be downloaded from a web-based system or anapplication which makes a video available for processing, in accordancewith the methods disclosed herein. The thermal image may also bereceived from an application such as those which are available forhandheld cellular devices and processed on the cell phone or otherhandheld computing devices such as an iPad or Tablet-PC. The thermalimage may be received directly from a memory or storage device of theimaging device that is used to capture that thermal image or a thermalvideo.

At step 304, the thermal image is obtained by selecting a single imageframe of a thermal video or a livestream thermal video. The thermalvideo or the livestream thermal video is captured using the thermalimaging camera. At step 306, the thermal image that meets the requiredview angle from one or more thermal images that are obtained from thethermal video oil live stream thermal video is selected using a taggingclassifier. The tagging classifier includes a machine learning modelthat determines the view of the thermal image and classifies the thermalimage as one of the discrete views such as a right lateral view, a rightoblique view, a frontal view, a left lateral view or a left oblique viewas per the thermal imaging protocol. At step 308, the selected thermalimages are provided to the adaptive view angle positioning system forfurther analysis.

FIG. 4 illustrates an exemplary process flow of an offline view angledetermination of a thermal image from a user using an adaptive viewangle positioning system according to an embodiment herein. At step 402,the thermal image is captured using a thermal imaging camera. At step404, the thermal image N uploaded into the adaptive view anglepositioning system. At step 406, the view angle of the uploaded thermal,image from a user is determined. At step 408, it is determined whetherthe thermal image is captured correctly by comparing the determined viewangle with a required view angle as per thermal imaging protocol whenthe thermal image does not meet the required view angle as per thermalimaging protocol to determine an angular adjustment to be made to a viewposition of the thermal imaging camera or a position of the subject. Atstep 410, the thermal image N provided for further analysis, for examplesegmentation and/or breast cancer screening, if the thermal image iscaptured with a required view angle as per the thermal imaging protocol.If not, at step 412 a set of instructions to the user is generated foradjusting the view position of the thermal imaging camera for capturinga new thermal image at the required view angle as per thermal imagingprotocol.

FIG. 5 illustrates an exemplary process flow of a live stream view angledetermination of a thermal image from a user using an adaptive viewangle positioning system according to an embodiment herein. At step 502,the livestream with sequential frames (e.g. thermal image/video) iscaptured using a thermal imaging camera. At step 504, a sample rate oradaptive sampling algorithms are applied to analyze selected frames ofthe thermal video instead of all frames to determine the view angle. Atstep 506, the view angle of the sampled thermal frame(s) from a user isdetermined. At step 508, it is determined whether the thermal frame(s)are captured correctly by comparing the determined view angles with arequired view angle as per thermal imaging protocol when the thermalframe(s) do not meet the required view angle as per thermal imagingprotocol to determine an angular adjustment to be made to a viewposition of the thermal imaging camera or a position of the subject. Atstep 510, the thermal framers) are provided for further analysis, forexample segmentation and/or breast cancer screening, if the thermalframe(s) are captured with a required view angle as per the thermalimaging protocol. If not, at step 512, a set of instructions to the useris generated for adjusting the view position of the thermal imagingcamera for capturing a new thermal image at the required view angle asper thermal imaging protocol.

FIG. 6 illustrates an exemplary process flow of a view angledetermination using an adaptive view angle positioning system to selecta view with adaptive sampling to reduce image frame candidates accordingto an embodiment herein. At step 602, the thermal video is capturedusing a thermal imaging camera. At step 604, the thermal video isuploaded to the adaptive view angle positioning system. At step 606, asample rate or adaptive sampling algorithms are applied to analyzeselected frames of the thermal video instead of all frames to determinethe view angle. At step 608, the view angles of the selected frames aredetermined. At step 610, it is determined whether the thermal framesmatch the requited view angles as per thermal imaging protocol bycomparing the determined view angles with the required vie angles as perthermal imaging protocol. At step 612, the thermal frames are providedfor further analysis, for example segmentation and/or breast cancerscreening, if the thermal frames are as per the thermal imagingprotocol. If not, it toes back to step 606 to adjust the sampling rate.

In an embodiment, the required thermal frames selected from a capturedthermal video of a subject (e.g. frames to be considered 0, ±45, ±90)are used for automated analysis. The input to the adaptive view anglepositioning system is the entire frames from the thermal video orsampled frame from any adaptive sampling algorithm. The adaptive viewangle positioning system determines the best frames corresponding to therequired view angles. In an embodiment, the adaptive view anglepositioning system determines the view angle of the thermal image from auser based on specific angles that is used for analysis such as breastcancer screening.

FIG. 7 illustrates a flow diagram of one embodiment of the presentmethod for determining a view angle of a thermal image from a user andgenerating a suggestion to enable the user for adaptive positioning of asubject for capturing the thermal image according to an embodimentherein. At step 702, a thermal image of a body of a subject is received.The thermal image represents the temperature distribution on the body ofthe subject as pixels in the thermal image with a highest temperaturevalue being displayed in a first color and pixels with a lowesttemperature value being displayed in a second color. pixels withtemperature values between the lowest and highest temperature valuesbeing displayed in gradations of color between the first and secondcolors. At step 704, a view angle of the thermal image from a user isautomatically determined using a view angle estimator. At step 706, anangular adjustment to be made to a view position of the thermal imagingcamera or a position of the subject is determined using a machinelearning model by comparing the determined view angle with a requiredview angle as per thermal imaging protocol when the thermal image doesnot meet the required view angle as per thermal imaging protocol. Atstep 708, a set of instructions is generated to the user for adjustingthe view position of the thermal imaging camera for capturing a newthermal image at the required view angle as per thermal imagingprotocol.

FIG. 8 illustrates a block diagram of one example adaptive view anglepositioning system/image processing system 800 for processing a thermalimage in accordance with the embodiments described with respect to theflow diagram of FIG. 7 according to an embodiment herein. Image Receiver802 wirelessly receives the video via antenna 801 having beentransmitted thereto from the video/thermal imaging device 101 of FIG. 1. Temperate Processor 803 performs a temperature-based method to detectpixels in the received image. View angle estimator 804 determines a viewangle of the thermal image front a user. Both Modules 803 and 804 storetheir results to storage device 805 Machine learning model 806 retrievesthe results of from storage device 805 and proceeds to determine anangular adjustment to be made to a view position of the thermal imagingcamera 101 or a position of the subject by comparing the determined viewangle with a required view angle as per thermal imaging protocol whenthe thermal image does not meet the required view angle as per thermalimaging protocol. The machine learning model 806 generates a set ofinstructions to the user for adjusting the view position of the thermalimaging camera 101 for capturing the new thermal image at the requiredview angle as per thermal imaging protocol. Central Processing Unit 808retrieves machine readable program instructions from a memory 809 and isprovided to facilitate the functionality of any of the modules of thesystem 800. CPU 808, operating alone or in conjunction with otherprocessors, may be configured to assist or otherwise perform thefunctionality of any of the modules or processing units of the system800 as well as facilitating communication between the system 800 and theworkstation 810.

System 800 is shown having been placed in communication with aworkstation 810. A computer case of the workstation houses variouscomponents such as a motherboard with a processor and memory, a networkcard, a video card a hard drive capable of reading/writing to machinereadable media 811 such as a floppy disk, optical disk, CD-ROM, DVD,magnetic tape, and the like, and other software and hardware needed toperform the functionality of a computer workstation. The workstationfurther includes a display device 812, such as a CRT, LCD, or touchscreen device, for displaying information, images, view angles, and thelike. A user can view any of that information and make a selection frommenu options displayed thereon. Keyboard 813 and mouse 814 effectuate auser input. It should be appreciated that the workstation has anoperating system and other specialized software configured to displayalphanumeric values, menus, scroll bars, dials, slideable bars,pull-down options, selectable buttons, and the like, for entering,selecting, modifying, and accepting information needed for processing inaccordance with the teachings hereof. The workstation is further enabledto display thermal images, view angle of the thermal images and the likeas they are derived. A user or technician may use the user interface ofthe workstation to set parameters, view/adjust the view angle, andadjust various aspects of the view angle estimation being performed, asneeded or as desired, depending on the implementation. Any of theseselections or inputs may be stored/retrieved to storage device 811.Default settings can be retrieved from the storage device. A user of theworkstation is also able to view or manipulate any of the data in thepatient records, collectively at 815, stored in database 816. Any of thereceived images, results, determined view angle, and the like, may bestored to a storage device internal to the workstation 810. Althoughshown as a desktop computer, the workstation can be a laptop, mainframe,or a special purpose computer such as an AMC, circuit, or the like.

Any of the components of the workstation may be placed in communicationwith any of the modules and processing units of system 800. Any of themodules of the system 800 can be placed in communication with storagedevices 805, 816 and 106 and/or computer readable media 811 and maystore/retrieve there from data, variables, records, parameters,functions, and/or machine readable/executable program instructions, asneeded to perform their intended functions. Each of the modules of thesystem 800 may be placed in communication with one or more remotedevices over network 817. It should be appreciated that some or all ofthe functionality performed by any of the modules or processing units ofthe system 800 can be performed, in whole or in part, by theworkstation. The embodiment shown is illustrative and should not beviewed as limiting the scope of the appended claims strictly to thatconfiguration. Various modules may designate one or more componentswhich may, in turn, comprise software and/or hardware designed toperform the intended function.

The foregoing description of the specific embodiments will so fullyreveal the general nature of the embodiments herein that others can, byapplying current knowledge, readily modify and/or adapt for variousapplications such specific embodiments without departing from thegeneric concept, and, therefore, such adaptations and modificationsshould and are intended to be comprehended within the meaning and rangeof equivalents of the disclosed embodiments. It is to be understood thatthe phraseology or terminology employed herein is for the purpose ofdescription and not of limitation. Therefore, while the embodimentsherein have been described in terms of preferred embodiments, thoseskilled in the art will recognize that the embodiments herein can bepracticed with modification within the spirit and scope.

We claim:
 1. A method for determining a view angle of a thermal imagefrom a user and generating a suggestion to enable the user for adaptivepositioning of a subject for capturing the thermal image, characterizedin that the method comprising: receiving a thermal image of a body of asubject, which represents the temperature distribution on the body ofthe subject as pixels in the thermal image with a highest temperaturevalue being displayed in a first color and pixels with a lowesttemperature value being displayed in a second color, pixels withtemperature values between the lowest and highest temperature valuesbeing displayed in gradations of color between the first and secondcolors, wherein the thermal image is captured by a thermal imagingcamera, wherein the thermal imaging camera comprising: an array ofsensors that converts infrared energy into electrical signals on aper-pixel basis; a lens that focuses the infrared energy from thesubject's body onto the array of sensors, wherein the array of sensorsdetects temperature values from the subject's body; and a specializedprocessor that processes the detected temperature values into at leastone block of pixels to generate the thermal image, automaticallydetermining a view angle of the thermal image from a user using a viewangle estimator; determining, using a machine learning model, an angularadjustment to be made to a view position of the thermal imaging cameraor a position of the subject by comparing the determined new angle witha required view angle as per thermal imaging protocol when the thermalimage does not meet the required view angle as per thermal imagingprotocol; and generating a set of instructions to the user for adjustingthe view position of the thermal imaging camera for capturing a newthermal image at the required view angle as per thermal imagingprotocol.
 2. The method of claim 1, wherein the method comprises step ofproviding the new captured thermal image along with determined viewangle for automatic breast segmentation followed by an automatic tumordetection method to detect cancerous tissue and/or non-cancerous tissuewithin a breast area of the subject.
 3. The method of claim 2, whereinthe method comprises step of implementing an automatic segmentingtechnique to segment the breast area of the subject in the thermal imageby: determining an outer side contour of an outline of a boundary of thebreast area of the subject from a body silhouette; determining an innerside boundary of the breast area from the body silhouette and the viewangle of the thermal image; determining an upper boundary of the breastarea by determining a lower boundary of an isotherm of axilla of thesubject; determining a lower boundary of the breast area by determiningan upper boundary of an isotherm of infra-mammary fold of the person;and segmenting the breast area of the subject by connecting abovedetermined breast boundaries to segment the breast from surroundingtissue in the thermal image.
 4. The method of claim 2, wherein theimplementation of the automatic segmenting technique to segment thebreast area of the subject in the thermal image comprises: training adeep learning model by providing a plurality of thermal images as aninput and the corresponding segmentation as an output to obtain thetrained deep learning model; and providing the new captured thermalimage to the trained deep learning model to predict a final segmentationmap.
 5. The method of claim 1, wherein the view angle estimatorcomprises a regression machine learning model that determines anapproximate view angle of the subject with respect to the camera usingthe thermal image.
 6. The method of claim 1, wherein the view angleestimator comprises a tagging classifier, wherein the tagging classifiercomprises a machine learning model that determines the view of thethermal image and classifies the thermal image as one of discrete viewssuch as a right lateral view, a right oblique view, a frontal view, aleft lateral view or a left oblique view as per the thermal imagingprotocol.
 7. The method of claim 1, wherein said set of instructions isprovided to at least one of a robotic arm holding the camera, anelectronically controlled camera stand or an electronically controlledrotating chair to automatically positions itself to the suggestedangular adjustment for capturing the new thermal image at the requiredview angle.
 8. The method of claim 1, wherein the view angle estimatorautomatically captures the new thermal image at the required view anglewithout user's intervention.
 9. The method of claim 3, wherein themethod comprises the step of displaying at least one of the determinedview angle of the thermal image or the segmented breast area on avisualization screen.
 10. The method of claim 2, wherein said automatictumor detection method comprises: determining a percentage of pixels p₁within said selected region of interest with a temperature T¹ _(pixel),where T₂≤T¹ _(pixel)≤T₁; determining a percentage of pixels p₂ withinsaid selected region of interest with a temperature T² _(pixel), whereT₃≤T² _(pixel); and determining a ratio p₃=P_(edge)/P_(block), whereP_(edge) is a number of pixels around a border of a suspected tumorwithin said region of interest, and P_(block) is a number of pixels inthe perimeter of said region of interest, wherein T₁, T₂ and T₃ aretemperature threshold obtained from temperature distribution.
 11. Themethod of claim 2, wherein the method comprises determining whether asuspected tumor region as one of: said cancerous tissue, saidnon-cancerous using a decision rule R, wherein said decision rule R isbased on any combination of: R₁, R₂, R₃, where: R₁=(p₁>Threshold₁),R₂=(p₂>Threshold₂), and R₃=(p₃>Threshold₃).
 12. The method of claim 1,wherein the view angle estimator automatically labels the thermal imagewith a label signifying its discrete view angle.
 13. The method of claim12, wherein the method comprises correcting view angle errors in saidthermal imaging protocol by modifying the data object propertiescomprising filenames using said label.
 14. The method of claim 1,wherein the view angle estimator is applied on a thermal age that isobtained by selecting a single image frame of a thermal video or a livestream thermal video, wherein the thermal video or the live streamthermal video is captured using the thermal imaging camera.
 15. Themethod of claim 14, wherein the view angle estimator comprises a taggingclassifier that selects a thermal image that meets the required viewangle from one or more thermal images that are obtained from a thermalvideo or live stream thermal video.
 16. A system for determining a viewangle of a thermal image from a user and generating a suggestion toenable the user for adaptive positioning of a subject for capturing thethermal image, characterized in that the system comprising: a storagedevice; and a processor retrieving machine-readable instructions fromthe storage device which, when executed by the processor, enable theprocessor to: receive a thermal image of a body of a subject, whichrepresents the temperature distribution on the body of the subject aspixels in the thermal image with a highest temperature value beingdisplayed in a first color and pixels with a lowest temperature valuebeing displayed in a second color, pixels with temperature valuesbetween the lowest and highest temperature values being displayed ingradations of color between the first and second colors, wherein thethermal image is captured by a thermal imaging camera, wherein thethermal imaging camera comprising: an array of sensors that convertsinfrared energy into electrical signals on a per-pixel basis; a lensthat focuses the infrared energy from the subject's body onto the arrayof sensors, wherein the array of sensors detects temperature values fromthe subject's body; and a specialized processor that processes thedetected temperature values into at least one block of pixels togenerate the thermal image; automatically determine a view angle of thethermal image from a user using a view angle estimator; determine, usinga machine learning n angular adjustment to be made to a view position ofthe thermal imaging camera or a position of the subject by comparing thedetermined view angle with a required view angle as per thermal imagingprotocol when the thermal image does not meet the required view angle asper thermal imaging protocol; and generate a set of instructions to theuser for adjusting the view position of the thermal imaging camera forcapturing a new thermal image at the required view angle as per thermalimaging protocol.
 17. The system of claim 16, wherein the systemprovides the new captured thermal image along with determined view angleto the view angle determination unit for automatic breast, segmentationfollowed by an automatic tumor detection method to detect canceroustissue and/or non-cancerous tissue within a breast area of the subject.18. The system of claim 17, wherein the system implements an automaticsegmenting technique to segment the breast area of the subject in thethermal image by: determining an outer side contour of an outline of aboundary of the breast area of the subject from a body silhouette;determining an inner side boundary of the breast area from the bodysilhouette and the view angle of the thermal image; determining an upperboundary of the breast area by determining a lower boundary of anisotherm of axilla of the subject; determining a lower boundary of thebreast area by determining an upper boundary of an isotherm ofinfra-mammary fold of the person; and segmenting the breast area of thesubject by connecting above determined breast boundaries to segment thebreast from surrounding tissue in the thermal image.
 19. The system ofclaim 17, wherein the system implements the automatic segmentingtechnique to segment the breast area of the subject in the thermal imageby: training a deep learning model by providing a plurality of thermalimages as an input and the corresponding segmentation as an output toobtain the trained deep learning model; and providing the new capturedthermal image to the trained deep learning model to predict a finalsegmentation map.
 20. The system of claim 16, wherein the systemprovides set of instructions to at least one of a robotic arm holdingthe camera, an electronically controlled camera stand or anelectronically controlled rotating chair to automatically positionsitself to the suggested angular adjustment for capturing the new thermalimage at the required view angle.